Peter Rudenko
Peter Rudenko
Can you please try with `export UCX_ERROR_SIGNALS=` on driver and `--conf spark.executorEnv.UCX_ERROR_SIGNALS=` in spark conf.
Can you please send `hs_err.log` file with whole stacktrace.
Does it work with the same parameters and not using ucx? Does it work with `-XX:+UseParallelGC`?
Does it happen at the beginning of the job, at map phase or reduce phase or at the end? Do you see something in `dmesg`? SparkUCX is doing memory mapping...
I'm using local cluster also, would be nice to provide public URL for dataset.