Where can I get the mmlspark python whl file？

[TomasZhang19]

Dear author， I use spark 2.3.2 and python 3.6.5. In the SynapseML README.md, 'SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+'. Could you please tell me where I can find or how to get the mmlspark python whl file for mmlspark-v1.0.0-rc1,or mmlspark-v0.18.1?

[github-actions[bot]]

Hey @TomasZhang19 :wave:! Thank you so much for reporting the issue/feature request :rotating_light:. Someone from SynapseML Team will be looking to triage this issue soon. We appreciate your patience.

[TomFinley]

Hello @TomasZhang19 . While building whl files has been a feature of this library since a bit over a year ago, the older versions you reference mmlspark 1.0.0.-rc1 did not have this capability. More specifically, the current facilities to even build the whl files were not added until #1223 (from a bit over a year ago). The ability for the build process to even create a whl is newer than the mmlspark version you're trying to use.

So: while that is a feature of the software currently, I'm sorry to report that it was not a feature of the software three years ago.

[TomFinley]

OK, I see that there are actually prior issues in #1767 and #1764 that explain more the intended use case. Note that it is possible to reply on an existing issue, rather than creating new ones, since these all appear to be the same core issue, that is, "how can I get the older version of the software to work on a Python 3.6/PySpark 2.3 environment."

Anyway, I explain how to use the software in #1767.

[TomasZhang19]

Hello @TomasZhang19 . While building whl files has been a feature of this library since a bit over a year ago, the older versions you reference mmlspark 1.0.0.-rc1 did not have this capability. More specifically, the current facilities to even build the whl files were not added until #1223 (from a bit over a year ago). The ability for the build process to even create a whl is newer than the mmlspark version you're trying to use.

So: while that is a feature of the software currently, I'm sorry to report that it was not a feature of the software three years ago.

@TomFinley Thang you for your reply. But according to #854 , for version mmlspark-0.17, there is a whl file that can be use: 'https://mmlspark.azureedge.net/pip/mmlspark-0.17-py2.py3-none-any.whl'. So I think there must be whl file for version mmlspark-v1.0.0-rc1 or mmlspark-v0.18.1. But I can't find them.

[TomasZhang19]

I run the command 'wget https://mmlspark.azureedge.net/pip/mmlspark-0.17-py2.py3-none-any.whl', and can download the file 'mmlspark-0.17-py2.py3-none-any.whl'. But I can't get the mmlspark-0.18.0-py2.py3-none-any.whl or mmlspark-0.18.1-py2.py3-none-any.whl in 'https://mmlspark.azureedge.net/pip' (ERROR 404: Not Found.). Does anyone know where to get them?

[TomFinley]

Well, there was an old script to generate a .whl that was removed over three years ago, which happened after the release of 0.17 and before the release of 0.18. So that isn't much help. Even in 0.17 to judge from the README.md at the time of 0.17 I don't see that this was a documented workflow. I can't dispute that the build scripts used to do this in versions of the software that are even older than 0.18, but by 0.18 they didn't. So that .whl file isn't there. And even when it was there, if it wasn't documented, it wasn't supported. You may note in that linked issue that the response leads with clarifying that the library is not intended to be used that way and this is not supported.

I have an idea. Maybe you could instead explain what effect you're trying to achieve, or at least try to explain why the supported pathways to using the software that I've explained don't work for you. I mean we could just keep going around like this and I could just keep saying "it wasn't supported then" or "that capability wasn't in the library three years ago" or "that isn't how you use this" but I'm not sure that brings you at all closer to what you're trying to do.

[TomasZhang19]

Well, there was an old script to generate a .whl that was removed over three years ago, which happened after the release of 0.17 and before the release of 0.18. So that isn't much help. Even in 0.17 to judge from the README.md at the time of 0.17 I don't see that this was a documented workflow. I can't dispute that the build scripts used to do this in versions of the software that are even older than 0.18, but by 0.18 they didn't. So that .whl file isn't there. And even when it was there, if it wasn't documented, it wasn't supported. You may note in that linked issue that the response leads with clarifying that the library is not intended to be used that way and this is not supported.

I have an idea. Maybe you could instead explain what effect you're trying to achieve, or at least try to explain why the supported pathways to using the software that I've explained don't work for you. I mean we could just keep going around like this and I could just keep saying "it wasn't supported then" or "that capability wasn't in the library three years ago" or "that isn't how you use this" but I'm not sure that brings you at all closer to what you're trying to do.

@TomFinley Thank you. I use a linux environment that can not visit Internet, so I need to install mmlspark using whl file. The spark version is 2.3.2 and python version is 3.6. According to the developer-readme.md, I can use version v0.17 ~ v1.0.0-rc1. Could you please tell me some other offline way to install mmlspark? The higher version is better.

[mhamilton723]

@TomasZhang19 is it possible to bump to spark 3? We no longer support spark 2.X so if you want to use the latest version and installation patterns we test regularly this is the simplest way