splash
splash copied to clipboard
MemVerge
Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange
Bumps hadoop-common from 2.7.4 to 3.2.3. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Hello, I have built and set up splash and spark according to the documentation. But when I try to run any spark application I am getting the following error message:...
Bumps commons-io from 2.4 to 2.7. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
In cloud environments, it is a common requirement to be able to persist shuffle data outside of the node on which a Spark task is running. Since many workloads run...
Allow the plugin developer to implement a fast merge which will be invoked when: * Encryption is disabled. * Compression is disabled or the compression codec supports concatenation of serialized...
Before build I've changed spark version in pom.xml: Spark-submit done in local[*] mode 2.4.0 java.lang.IllegalAccessError: tried to access class org.apache.spark.shuffle.sort.ShuffleInMemorySorter from class org.apache.spark.shuffle.sort.SplashUnsafeSorter at org.apache.spark.shuffle.sort.SplashUnsafeSorter.(SplashUnsafeSorter.scala:77) at org.apache.spark.shuffle.sort.SplashUnsafeShuffleWriter.(SplashUnsafeShuffleWriter.scala:56) at org.apache.spark.shuffle.SplashShuffleManager.getWriter(SplashShuffleManager.scala:84) at...
The output of the shuffle performance tool is not precise enough. For example, 1.23GB would only be displayed as 1GB. The output should keep at least two digits for the...
Bumps commons-io:commons-io from 2.4 to 2.14.0. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
Bumps org.apache.hadoop:hadoop-common from 2.7.4 to 3.4.0. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...