Support Java 11 and add trino plugin

[jles01]

Supoort for java 11 and trino plugin

[exolab]

Is this still being worked on?

[jles01]

Hi,

It is working, there will be minor changes, to make it build with success. Also in other PR there will be added Trino support for Hive security, so they can be reused

[adnanhemani]

Hi, isn't the current presto plugin the one that should work for Trino as well? I understand after the Trino name change, there may need to be some package name changing, but for the most part there doesn't need to be any logical difference or use for the old "presto" plugin. Shouldn't we just rename everything in the Presto plugin and make it the official support for Trino?

[jles01]

Hi, isn't the current presto plugin the one that should work for Trino as well? I understand after the Trino name change, there may need to be some package name changing, but for the most part there doesn't need to be any logical difference or use for the old "presto" plugin. Shouldn't we just rename everything in the Presto plugin and make it the official support for Trino?

From my knowledge presto still exists and will be continued. Should we move that users outside apache ranger ? I would not be so sure. We can have both or only Trino, the question would be how many "presto ranger" users out there

[adnanhemani]

From what I gathered, it seemed like PrestoDB was not supported by our current presto plugin and that a new PrestoDB plugin would be required that would have slightly different logic. If that's correct, I'd think that it makes more sense to say the current Ranger version (v2.1.0) is the latest version supported for all Trino versions before the renaming and the next Ranger version will have support for all Trino versions past the Trino renaming - and that we should change the current presto plugin to be the official Trino plugin

[adnanhemani]

Actually, I just read this thread: here. Seems like a new Trino plugin will be required - but not as a result of the issues we were discussing 😁

Watching thread for progress.

[jles01]

Ok so the new plugin is all set up :) the question should I remove the presto one or not ...

[adnanhemani]

I think we can keep it for now 🤷 we can remove in a later PR, if required?

[jerryleooo]

Encountered following error

+ Fri Oct  8 18:21:12 +08 2021 : Saving lib file: /opt/presto/lib/validation-api-2.0.1.Final.jar to /opt/presto/lib/.validation-api-2.0.1.Final.jar.20211008182112 ...
Unable to store password in non-plain text format. Error: [SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang3/StringUtils
        at org.apache.hadoop.metrics2.lib.MutableMetricsFactory.getName(MutableMetricsFactory.java:134)
        at org.apache.hadoop.metrics2.lib.MutableMetricsFactory.getInfo(MutableMetricsFactory.java:130)
        at org.apache.hadoop.metrics2.lib.MutableMetricsFactory.newForField(MutableMetricsFactory.java:45)
        at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder.add(MetricsSourceBuilder.java:147)
        at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder.<init>(MetricsSourceBuilder.java:69)
        at org.apache.hadoop.metrics2.lib.MetricsAnnotations.newSourceBuilder(MetricsAnnotations.java:43)
        at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:223)
        at org.apache.hadoop.metrics2.MetricsSystem.register(MetricsSystem.java:71)
        at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:149)
        at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:265)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3614)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3604)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3441)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
        at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
        at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
        at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:74)
        at org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:87)
        at org.apache.ranger.credentialapi.buildks.main(buildks.java:41)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:606)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:168)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 24 more]
Exiting plugin installation

Java version: 15.0.1

[preetsindhal]

Hi, can anyone share how to build the Trino plugin with 2.3 release i have tried the following but no tar file was created for trino plugin.

mvn -Pall -DskipTests=true clean compile package install

mvn -Pranger-jdk11 -DskipTests=true clean compile package install

mvn -Pranger-jdk11 -DskipTests=true clean compile package install assembly:assembly - fails with error

I tried building it as explained here https://issues.apache.org/jira/browse/RANGER-3755 but no luck, getting the same errors as last comment on this jira issue.

thanks.

[esselius]

Hi, can anyone share how to build the Trino plugin with 2.3 release i have tried the following but no tar file was created for trino plugin.

mvn -Pall -DskipTests=true clean compile package install

mvn -Pranger-jdk11 -DskipTests=true clean compile package install

mvn -Pranger-jdk11 -DskipTests=true clean compile package install assembly:assembly - fails with error

I tried building it as explained here issues.apache.org/jira/browse/RANGER-3755 but no luck, getting the same errors as last comment on this jira issue.

thanks.

I finally figured out how to do this, turns out you need to disable the distro linux profile and pick a few modules manually to make it work:

$ cd apache-ranger-2.4.0
$ mvn clean package -DskipTests -P ranger-trino-plugin,-linux -am -pl distro,plugin-trino,ranger-trino-plugin-shim,agents-installer,credentialbuilder

The artifact lands @ target/ranger-2.4.0-trino-plugin.tar.gz

The problem was that the linux profile in the distro module implicitly activates and overrides the ranger-trino-plugin profile maven config etc

[dshershov]

Hi @esselius ! Your solution is very close to what I need! Unfortunately this is not working for me on 2.4.0 ranger version with 17ojdk. Maybe you have some ideas how I can fix that. I tried build trino plugin with 414 version of trino with the next command: JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home && mvn clean package -DskipTests -P ranger-trino-plugin,-linux(or -mac or -"mac os x") -am -pl agents-audit,agents-common,agents-cred,agents-installer,credentialbuilder,ranger-plugin-classloader,ranger-util,plugin-trino,ranger-trino-plugin-shim

[INFO] Reactor Summary for ranger 2.4.0:
[INFO]
[INFO] ranger ............................................. SUCCESS [  0.897 s]
[INFO] Credential Support ................................. SUCCESS [  1.974 s]
[INFO] Audit Component .................................... SUCCESS [  2.799 s]
[INFO] ranger-plugin-classloader .......................... SUCCESS [  0.906 s]
[INFO] Common library for Plugins ......................... SUCCESS [  5.663 s]
[INFO] Installer Support Component ........................ SUCCESS [  0.513 s]
[INFO] Credential Builder ................................. SUCCESS [  1.403 s]
[INFO] Ranger Util ........................................ SUCCESS [  1.196 s]
[INFO] Trino Security Plugin .............................. SUCCESS [  2.398 s]
[INFO] Trino Security Plugin Shim ......................... SUCCESS [  1.300 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

But archive was missing into target directory. Thanks a lot for your ideas!

[tarekabouzeid]

Hi @esselius ! Your solution is very close to what I need! Unfortunately this is not working for me on 2.4.0 ranger version with 17ojdk. Maybe you have some ideas how I can fix that. I tried build trino plugin with 414 version of trino with the next command: JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home && mvn clean package -DskipTests -P ranger-trino-plugin,-linux(or -mac or -"mac os x") -am -pl agents-audit,agents-common,agents-cred,agents-installer,credentialbuilder,ranger-plugin-classloader,ranger-util,plugin-trino,ranger-trino-plugin-shim

[INFO] Reactor Summary for ranger 2.4.0:
[INFO]
[INFO] ranger ............................................. SUCCESS [  0.897 s]
[INFO] Credential Support ................................. SUCCESS [  1.974 s]
[INFO] Audit Component .................................... SUCCESS [  2.799 s]
[INFO] ranger-plugin-classloader .......................... SUCCESS [  0.906 s]
[INFO] Common library for Plugins ......................... SUCCESS [  5.663 s]
[INFO] Installer Support Component ........................ SUCCESS [  0.513 s]
[INFO] Credential Builder ................................. SUCCESS [  1.403 s]
[INFO] Ranger Util ........................................ SUCCESS [  1.196 s]
[INFO] Trino Security Plugin .............................. SUCCESS [  2.398 s]
[INFO] Trino Security Plugin Shim ......................... SUCCESS [  1.300 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

But archive was missing into target directory. Thanks a lot for your ideas!

Hi,

Did you manage to get it to work with this Trino version?

Thanks