apache
HIVE-27855: Create external tables for Hive and Tez protologging events
What changes were proposed in this pull request?
HiveProtoLoggingHook in Hive and ProtoHistoryLoggingService in Tez logs details, such as query execution, query plan, runtime statistics, serialised as protobuf files. This PR exposes these protobuf log files as external tables. This would be done during the information schema upgrade to the next Hive version, which is 4.0.0-beta-2 in this case.
Why are the changes needed?
By exposing the protobuf log files as external tables, it can be useful for debugging, performance tuning and auditing purposes.
Does this PR introduce any user-facing change?
No
Is the change a dependency upgrade?
No
How was this patch tested?
This patch was tested by upgrading the information schema version and validating that the external tables have been created pointing to the correct specified locations.
Quality Gate passed
The SonarCloud Quality Gate passed, but some issues were introduced.
4 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication
See analysis details on SonarCloud
![sonarqubecloud[bot] avatar](https://avatars.githubusercontent.com/in/12526?v=4 "Published by a pub.dev verified publisher"){kind=small}
Quality Gate passed
Issues
3 New issues
0 Accepted issues
Measures
0 Security Hotspots
No data about Coverage
No data about Duplication
See analysis details on SonarCloud
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
No data about Coverage
No data about Duplication
See analysis details on SonarCloud
We may need to fix some tests
In the PR commit build I am getting this error, but I have not changed anything in TestMiniLlapLocalCliDriver. My changes are only related to schema and I'm unable to understand as to why this specific test is failing. Could you please help me with fixing the build? Thank you.
./itests/qtest/target/surefire-reports/TEST-org.apache.hadoop.hive.cli.split23.TestMiniLlapLocalCliDriver.xml:59905.176: internal error: Huge input lookup
In the PR commit build I am getting this error, but I have not changed anything in TestMiniLlapLocalCliDriver. My changes are only related to schema and I'm unable to understand as to why this specific test is failing. Could you please help me with fixing the build? Thank you.
I think at least these failed because of this PR.
- https://ci.hive.apache.org/blue/rest/organizations/jenkins/pipelines/hive-precommit/branches/PR-5036/runs/7/nodes/667/log/?start=0
- https://ci.hive.apache.org/blue/rest/organizations/jenkins/pipelines/hive-precommit/branches/PR-5036/runs/7/nodes/514/log/?start=0
- https://ci.hive.apache.org/blue/rest/organizations/jenkins/pipelines/hive-precommit/branches/PR-5036/runs/7/nodes/502/log/?start=0
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
See analysis details on SonarCloud