bigtop icon indicating copy to clipboard operation
bigtop copied to clipboard

BIGTOP-3928. Fix build failure of Hive due to unresolved transitive dependency on javax.el.

Open iwasakims opened this issue 1 year ago • 4 comments

https://issues.apache.org/jira/browse/BIGTOP-3928

./gradlew hive-pkg fails due to unresolved dependency on javax.el.

[ERROR] Failed to execute goal on project hive-hbase-handler: Could not resolve dependencies for project org.apache.hive:hive-hbase-handler:jar:3.1.3: Failed to collect dependencies at org.apache.hbase:hbase-server:jar:tests:2.4.13 -> org.glassfish.web:javax.servlet.jsp:jar:2.3.2 -> org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Failed to read artifact descriptor for org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Could not transfer artifact org.glassfish:javax.el:pom:3.0.1-b06-SNAPSHOT from/to jvnet-nexus-snapshots (https://maven.java.net/content/repositories/snapshots): Transfer failed for https://maven.java.net/content/repositories/snapshots/org/glassfish/javax.el/3.0.1-b06-SNAPSHOT/javax.el-3.0.1-b06-SNAPSHOT.pom: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -> [Help 1]

Applying backport patch of HIVE-19579 fixed this.

iwasakims avatar Apr 12 '23 10:04 iwasakims

ran smoke-tests of hive on Rocky Linux 8 (x86_64).

iwasakims avatar Apr 12 '23 11:04 iwasakims

Thanks for working on it, @iwasakims I verified the Haddop and Hive smoke tests on Centos-7 and Fedora-36 successfully (x86_64 / Arm64).

However, when testing on Ubuntu 22.04, the deployment of Hive failed due to dependency issues.

config_ubuntu-22.04.yaml :

...
components: [bigtop-utils, zookeeper,hdfs, yarn, mapreduce, hive]
enable_local_repo: true
smoke_test_components: [hdfs, yarn, mapreduce, hive]

./docker-hadoop.sh -d -C config_ubuntu-22.04.yaml -c 3 -s

Log:

Notice: /Stage[main]/Hadoop::Client/Package[hadoop-hdfs-fuse]/ensure: created
Notice: /Stage[main]/Hadoop::Client/Package[libhdfs0-dev]/ensure: created
Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install hive' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 hive : Depends: python but it is not installable
E: Unable to correct problems, you have held broken packages.
Error: /Stage[main]/Hadoop_hive::Client_package/Package[hive]/ensure: change from 'purged' to 'latest' failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install hive' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 hive : Depends: python but it is not installable
E: Unable to correct problems, you have held broken packages.

guyuqi avatar Apr 19 '23 08:04 guyuqi

However, when testing on Ubuntu 22.04, the deployment of Hive failed due to dependency issues.

@guyuqi It can be reproduced regardless of the patch. I filed BIGTOP-3930. All packages depending on python should be affected. Let's resolve BIGTOP-3920 and this first.

iwasakims avatar Apr 19 '23 12:04 iwasakims

The deb packages worked on Debian 11 at least.

iwasakims avatar Apr 19 '23 16:04 iwasakims