s3committer
s3committer copied to clipboard
Missing dependency nebula.javadoc-jar
running a gradle build will crash since its missing a dependency.
Updating the build.gradle with
buildscript {
repositories {
jcenter()
maven {
url "https://plugins.gradle.org/m2/"
}
}
dependencies {
classpath 'com.github.jengelman.gradle.plugins:shadow:1.2.3'
classpath "com.netflix.nebula:nebula-publishing-plugin:4.9.1"
}
}
Seems to fix this
I really want to test this in our Apache Spark setup. We are really bottle necking on the FileOutputCommitter when writing to S3 and we need to write quite a lot of data.
Any idea on how the fix the issues to that I can build the jar, or where I can find a jar that already has been build?
Regards, Jork
@jorkzijlstra For me it worked to just remove this
apply plugin: 'nebula.javadoc-jar'
apply plugin: 'nebula.source-jar'
from the gradle build file. There is also an open pull request solving the issue (i.e. you could just check out that fork): https://github.com/rdblue/s3committer/pull/4
Can you open a PR with the fix?