mrjob
mrjob copied to clipboard
python_bin can't take arguments on Spark
As noted in #1376, PYSPARK_PYTHON
may not take arguments. This means running Spark scripts with something like --python-bin 'python -v'
won't work.
This can be solved by using a Python wrapper script; see #1376.