Jim Kleckner

Rebased to master.

Rebased to master

An interesting alternative in the `Scala` language is `zio` [1]. It uses `Structured Concurrency` and makes this statement [2]: > ZIO has support for structured concurrency. The way ZIO structured...

We haven't yet turned on istio in spark pods (have others with it) but we're looking at scuttle [1] as an interesting workaround for istio sidecars. Also take a look...

This might be related? JFrog to Shut down JCenter and Bintray https://www.infoq.com/news/2021/02/jfrog-jcenter-bintray-closure/

Searching around the code and docs, I don't see it documented. It seems that wanting to have multiple spark operators installed in a development cluster is something others want to...

Ephemeral storage is usually limited the amount of local disk on the node. Check that.

It can be either running out of space on the node or the request limit for your pod. I have seen situations where we needed to limit a pod from...

Also note this cryptic comment that the `SPARK_WORKER_DIR` can be used to alter the volume where ephemeral data is stored. [1] https://github.com/apache/spark/commit/71fc11357420b2207d1b29ab4502987293565a67

I look at the local (ephemeral) disk usage on executor nodes and try to keep the peak of it during a day < 15% or so for headroom. Also, on...