mixed case appName in Spark cluster fails on submission

[willferreira]

From the examples directory:

aztk spark cluster submit --id <cluster> --name PythonPi pi.py

causes:

(snip)
raise AzureHttpError(message, http_error.status)
azure.common.AzureHttpError: The specifed resource name contains invalid characters.
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:618773ac-001e-00d3-1bbd-726743000000
Time:2017-12-11T20:23:53.1187632Z</Message></Error>

But if I edit pi.py so that:

spark = SparkSession.builder.appName("pythonpi").getOrCreate()

aztk spark cluster submit --id <cluster> --name pythonpi pi.py

doesn't fail (in the same way). The cluster was created by:

aztk spark cluster create --id <cluster> --vm-size standard_a2 --size 2 --docker-repo aztk/python:spark2.2.0-python3.6.2-base

[jafreck]

Thanks for bringing this up. This is a limitation with Azure storage naming conventions, which we take a dependency on. We have a related open issue on this, here.

That error message is not very helpful at all -- so we will make sure to do some proper validation and give better feedback.

In the specific case of camel case naming, we might be able to get away lower casing the name instead of failing.

[willferreira]

Ok, thanks. I didn't spot the related open issue. It's not difficult to find a workaround, but the examples should be changed, since they won't work as written - they all have mixed-case appNames.