Geoff Duniam

thanks for the quick reply - in the meantime, worthwhile trying on Spark 1.6?

Brilliant, thanks - I'll keep an eye out.

Hi Derek, Ran up the tests on Spark 2.3.0, still getting the same issues, even on small numbers of iterations. FYI, the infrastructure we're running is Cloudea CDH 5.16.1 on...

Hi Derek, Thanks for the quick response - we were running tensorflow 1.13 so we've downgraded to tensorflow 1.7; looks like that may have sorted the issue. I'll be testing...

Hi Derek and Howanu, Derek, I'll echo what Howanu said, that code clears it up nicely. I've gone through the YARN and Spark settings, turns out that the yarn.nodemanager.resource settings...

Howanu, just a thought - we're running multiple virtual environments against Spark (Python 2.7 and 3.6 against Spark 1.6 and 2.3) and that seems to work quite well? We're running...

Hi Derek, Great news - any idea when the fix is available? We are currently using Python 2.7 for some of our analysis but frankly I don't see that as...

Will do - just finishing up some testing and we'll be installing the new version later today. I'll keep you posted. Thanks again :) Geoff

Hi Derek, Ugraded to 0.7.0, but the logs are showing this error - py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. : org.apache.spark.SparkException: Only one SparkContext may be running in this...

Hi Derek, The DNN process seems to be throwing the error- monitoring the job on YARN, runs fine up until the process runs the fit statement, and then the process...