Mobius
Mobius copied to clipboard
Local Environment Spark + SparkCLR Installer?
I've been trying to get into Spark, using Mobius as a bridge since I am primarily a .NET developer, and my company uses Microsoft products through most of the stack. Unfortunately, the process to get an environment to even run a word count is very cumbersome and error prone, as it involves much more than just adding a nuget package. Can there be an MSI installer or Chocolatey package available? Even SQL Server is a much easier process to get started in a Dev environment.
You can give this a try - wrote it for Spark 1.6.2: https://github.com/petabridge/spark-installer-windows
Unfortunately, this package no longer works, even in the older version once published. ` C:\WINDOWS\system32>choco install apache-spark -version 1.6.2 Parsing -version resulted in error (converted to warning): Cannot bundle unregistered option '-e'. Chocolatey v0.10.5 Installing the following packages: apache-spark By installing you accept licenses for the packages. apache-spark not installed. The package was not found with the source(s) listed. If you specified a particular version and are receiving this message, it is possible that the package name exists but the version does not. Version: "1.6.2" Source(s): "https://chocolatey.org/api/v2/"
Chocolatey installed 0/1 packages. 1 packages failed. See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Failures
- apache-spark - apache-spark not installed. The package was not found with the source(s) listed. If you specified a particular version and are receiving this message, it is possible that the package name exists but the version does not. Version: "1.6.2" Source(s): "https://chocolatey.org/api/v2/" `
Welp, that sucks.