Carson Wang

@YeahNew , JDK 1.8 is fine. It looks like a version mismatch issue as I saw a `NoSuchMethodError`. Just to check, did you upgrade Ray and RayDP on every node...

Confirmed with @YeahNew offline, the error only happens when running in IDE. There is no error if running in command line.

@ssiegel95 , can you please check if you have Java installed and JAVA_HOME set properly? can you also try to run pyspark to see if you encounter the same error?

It seems this is related to a bug in scala. But not sure when this could happen. I've upgraded scala to 2.12.14. please see if the new nightly release will...

@ssiegel95 I forgot to mention the new nightly release is still not available yet. It will be uploaded automatically every day if there are new commit. Anyway, I'll take a...

@ssiegel95 , I was able to reproduce the issue. It is an issue with Ray 1.4.1. You can use Ray 1.4.0 to bypass the issue for now.

Hi @YeahNew , it seems this is no longer related to Spark. If your data is not in Spark, probably you can directly use Ray's API instead of APIs from...

@Hoeze ,thanks for the feedback and update!

@SrilekhaIG , thanks for the feedback. I suspect if the SparkNLP jar file is properly included in the driver/executor classpath correctly when running on RayDP. I'll take a look. Your...

@SrilekhaIG , there is a problem in the executor classpath when using `spark.jars.packages` in RayDP. As a workaround, can you please try to install latest raydp-nightly and set "raydp.executor.extraClassPath" to...