flink-on-k8s-operator icon indicating copy to clipboard operation
flink-on-k8s-operator copied to clipboard

Published 20 hours ago • verified publisher icon GoogleCloudPlatform

Support classpath files

Open Mrart opened this issue 5 years ago • 7 comments

Flink command like this“-C,--classpath ” -C,--classpath <url> Adds a URL to each user code classloader on all nodes in the cluster. The paths must specify a protocol (e.g. file://) and be accessible on all nodes (e.g. by means of a NFS share). You can use this option multiple times for specifying more than one URL. The protocol must be supported by the {@link java.net.URLClassLoader}. https://ci.apache.org/projects/flink/flink-docs-stable/ops/cli.html

Mrart avatar Feb 26 '20 10:02 Mrart

I think we can add a field in the job spec.

functicons avatar Feb 26 '20 17:02 functicons

Ok, I will do it.

Mrart avatar Feb 27 '20 03:02 Mrart

@functicons --classpath param not only support jar file, It can support other properties files, So I have change the tittle.

Mrart avatar Feb 29 '20 04:02 Mrart

Before do it ， must fixed flink client bug to load classpath file error

Mrart avatar Mar 06 '20 12:03 Mrart

What do you mean by "flink client bug"?

functicons avatar Mar 07 '20 20:03 functicons

I have a ML discuss for [email protected]. flink client need --classpath param before -class to load the classpath. Maybe it will fixed.

Mrart avatar Mar 09 '20 02:03 Mrart

Does --classpath work on YARN? I am using Flink 1.12 on YARN cluster but was never able to resolve file:///path/to/hbase-client.jar using --classpath even though the option --classpath was specified before -c option for class name. Finally, I had to revert to creating symlink to my hbase-client.jar file in $FLINK_HOME/lib directory. This is not ideal approach but not sure if we need any configuration for --classpath option to work.

The same issue I am facing for loading hbase-site.xml for Phoenix Job.

piyushpatel2005 avatar Apr 18 '22 23:04 piyushpatel2005