Fix selectSparkVersion() when sparkVersion is provided

[Etherdaemon]

Changes

The current implementation of selectSparkVersion() is broken when sparkVersion is supplied. This is due to the code requiring an exact match instead of a contains.

The referenced Go code it was copied from uses contains not equals as well, and when you check the returned payload you can see it would not currently find any matching spark versions.

➜ curl -H "Authorization: Bearer $<token>" https://<workspace>/api/2.0/clusters/spark-versions
{"versions":[{"key":"12.2.x-scala2.12","name":"12.2 LTS (includes Apache Spark 3.3.2, Scala 2.12)"},{"key":"11.3.x-photon-scala2.12","name":"11.3 LTS Photon (includes Apache Spark 3.3.0, Scala 2.12)"},{"key":"14.2.x-cpu-ml-scala2.12","name":"14.2 ML (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"12.2.x-aarch64-photon-scala2.12","name":"12.2 LTS Photon aarch64 (includes Apache Spark 3.3.2, Scala 2.12)"},{"key":"10.4.x-cpu-ml-scala2.12","name":"10.4 LTS ML (includes Apache Spark 3.2.1, Scala 2.12)"},{"key":"9.1.x-aarch64-scala2.12","name":"9.1 LTS aarch64 (includes Apache Spark 3.1.2, Scala 2.12)"},{"key":"14.2.x-gpu-ml-scala2.12","name":"14.2 ML (includes Apache Spark 3.5.0, GPU, Scala 2.12)"},{"key":"9.1.x-photon-scala2.12","name":"9.1 LTS Photon (includes Apache Spark 3.1.2, Scala 2.12)"},{"key":"10.4.x-scala2.12","name":"10.4 LTS (includes Apache Spark 3.2.1, Scala 2.12)"},{"key":"13.3.x-scala2.12","name":"13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)"},{"key":"11.3.x-cpu-ml-scala2.12","name":"11.3 LTS ML (includes Apache Spark 3.3.0, Scala 2.12)"},{"key":"11.3.x-scala2.12","name":"11.3 LTS (includes Apache Spark 3.3.0, Scala 2.12)"},{"key":"13.3.x-cpu-ml-scala2.12","name":"13.3 LTS ML (includes Apache Spark 3.4.1, Scala 2.12)"},{"key":"10.4.x-photon-scala2.12","name":"10.4 LTS Photon (includes Apache Spark 3.2.1, Scala 2.12)"},{"key":"14.3.x-photon-scala2.12","name":"14.3 LTS Photon (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"13.3.x-aarch64-scala2.12","name":"13.3 LTS aarch64 (includes Apache Spark 3.4.1, Scala 2.12)"},{"key":"14.1.x-scala2.12","name":"14.1 (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"14.3.x-cpu-ml-scala2.12","name":"14.3 LTS ML (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"9.1.x-scala2.12","name":"9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12)"},{"key":"12.2.x-photon-scala2.12","name":"12.2 LTS Photon (includes Apache Spark 3.3.2, Scala 2.12)"},{"key":"12.2.x-cpu-ml-scala2.12","name":"12.2 LTS ML (includes Apache Spark 3.3.2, Scala 2.12)"},{"key":"14.2.x-photon-scala2.12","name":"14.2 Photon (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"11.3.x-aarch64-scala2.12","name":"11.3 LTS aarch64 (includes Apache Spark 3.3.0, Scala 2.12)"},{"key":"14.0.x-scala2.12","name":"14.0 (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"14.0.x-gpu-ml-scala2.12","name":"14.0 ML (includes Apache Spark 3.5.0, GPU, Scala 2.12)"},{"key":"11.3.x-gpu-ml-scala2.12","name":"11.3 LTS ML (includes Apache Spark 3.3.0, GPU, Scala 2.12)"},{"key":"14.0.x-photon-scala2.12","name":"14.0 Photon (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"10.4.x-aarch64-photon-scala2.12","name":"10.4 LTS Photon aarch64 (includes Apache Spark 3.2.1, Scala 2.12)"},{"key":"9.1.x-cpu-ml-scala2.12","name":"9.1 LTS ML (includes Apache Spark 3.1.2, Scala 2.12)"},{"key":"14.3.x-scala2.12","name":"14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"11.3.x-aarch64-photon-scala2.12","name":"11.3 LTS Photon aarch64 (includes Apache Spark 3.3.0, Scala 2.12)"},{"key":"14.1.x-gpu-ml-scala2.12","name":"14.1 ML (includes Apache Spark 3.5.0, GPU, Scala 2.12)"},{"key":"10.4.x-aarch64-scala2.12","name":"10.4 LTS aarch64 (includes Apache Spark 3.2.1, Scala 2.12)"},{"key":"9.1.x-gpu-ml-scala2.12","name":"9.1 LTS ML (includes Apache Spark 3.1.2, GPU, Scala 2.12)"},{"key":"apache-spark-2.4.x-scala2.11","name":"Light 2.4 (includes Apache Spark 2.4, Scala 2.11)"},{"key":"13.3.x-gpu-ml-scala2.12","name":"13.3 LTS ML (includes Apache Spark 3.4.1, GPU, Scala 2.12)"},{"key":"14.3.x-gpu-ml-scala2.12","name":"14.3 LTS ML (includes Apache Spark 3.5.0, GPU, Scala 2.12)"},{"key":"14.1.x-cpu-ml-scala2.12","name":"14.1 ML (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"14.2.x-scala2.12","name":"14.2 (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"13.3.x-aarch64-photon-scala2.12","name":"13.3 LTS Photon aarch64 (includes Apache Spark 3.4.1, Scala 2.12)"},{"key":"14.0.x-cpu-ml-scala2.12","name":"14.0 ML (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"12.2.x-gpu-ml-scala2.12","name":"12.2 LTS ML (includes Apache Spark 3.3.2, GPU, Scala 2.12)"},{"key":"13.3.x-photon-scala2.12","name":"13.3 LTS Photon (includes Apache Spark 3.4.1, Scala 2.12)"},{"key":"10.4.x-gpu-ml-scala2.12","name":"10.4 LTS ML (includes Apache Spark 3.2.1, GPU, Scala 2.12)"},{"key":"14.1.x-photon-scala2.12","name":"14.1 Photon (includes Apache Spark 3.5.0, Scala 2.12)"},{"key":"12.2.x-aarch64-scala2.12","name":"12.2 LTS aarch64 (includes Apache Spark 3.3.2, Scala 2.12)"}]}%

Tests

Ran unit tests

[mgyucht]

Thanks for raising this! Can you add a unit test to ClustersExtTest to test this behavior? Otherwise this is fine.