Don't require spark-env.sh configuration file to be present

Open JoshRosen opened this issue 11 years ago • 2 comments

Spark doesn't require this file, so we shouldn't throw an error message if it's not present when we try to copy it.

Nov 14 '14 21:11 JoshRosen

Also: don't require the slaves file: in 1.2+, the absence of a slaves file causes us to just run a cluster with a single local worker. We still need this file for backwards-compatibility with earlier Spark versions, so we should probably log a warning if it's missing instead of throwing an error.

This wouldn't be an issue if we had separate configurations for local and cluster mode (#22).

Nov 17 '14 20:11 JoshRosen

The slaves file issue was fixed by #29, but I think we still have to fix the spark-env.sh one.

Nov 25 '14 22:11 JoshRosen