Universal plugin fails with java.io.IOException: Error writing to server

[dotta]

Expected behaviour

sbt universal:publish should publish a zip file to the set repository

Actual behaviour

sbt universal:publish fails with a java.io.IOException: Error writing to server error

Information

• What sbt-native-packager are you using 1.2.2 (also tried with 1.2.1 and 1.2.0)
• What sbt version 0.13.16 (also tried 0.13.13)
• What is your build system (e.g. Ubuntu, MacOS, Windows, Debian ) MacOS
• What package are you building (e.g. docker, rpm, ...) zip

Reproducible

To reproduce the issue, you must to use a private artifactory repository.

build.sbt

val CustomResolver = "My custom resolver" at "<url>" // <- provide the url to the private repository

lazy val root = (project in file("."))
  .settings(
    publishArtifact := false,
    publish := ()
  )
  .aggregate(sub)

lazy val sub = (project in file("sub"))
  .enablePlugins(JavaAppPackaging, UniversalDeployPlugin) // comment JavaAppPackaging and it will work!
  .settings(zippedDistPublishSettings)
  .settings(
    organization := "com.github.dotta",
    name := "test",
    scalaVersion := "2.12.3",
    version := "0.0.1-SNAPSHOT",
    credentials += Credentials(Path.userHome / ".ivy2" / ".credentials") // <-- credentials to publish to the configured private repo
  )


def zippedDistPublishSettings = Seq(
  resolvers += CustomResolver,
  publishTo in Universal := Some(CustomResolver),
  name in Universal := name.value + "_" + scalaVersion.value
)

Steps to reproduce it

Just type sbt universal:publish and you'll get

[error] (sub/universal:publish) java.io.IOException: Error writing to server

Now remove JavaAppPackaging from the build file. Try again sbt universal:publish and this time it will work! From the outside, it looks like the JavaAppPackaging plugin is doing something with the credentials, but I haven't been able yet to validate this hunch.

If you do need to use JavaAppPackaging, the only workaround I found to make it work is to do the following:

1 - type sbt and enter the interactive shell 2 - remove the JavaAppPackaging from the sbt subproject that's failing 3 - type reload 4 - type universal:publish (this will publish a zip that is very likely not what you want to publish) 5 - add back JavaAppPackaging 6 - type reload 7 - type universal:publish (this time you are publishing the intended zip)

This is far from being an actual workaround, but it serves displaying some unintended (?) side-effects happening when removing/adding JavaAppPackaging.

[muuki88]

Thanks for your report. I'll take a look when I'm back from my vacation.

[dotta]

Hey! I'm actually trying to understand what's happening and find at least a proper workaround, if not actually fixing the underlying issue. This to me smells like "scoping problem". Let me know if you have some idea for things that could be responsible for this!

[muuki88]

Thanks for taking the time 😍

A good starting point would be to create a new test based on this one.

Most of these weird issues are related to scoping from my experience so far. I must admit that I haven't touched the deploy plugins in anyway. They are still pretty old.

If you like we can have a short chat or hangout/Skype call if you like.

[dotta]

@muuki88 Thanks for chiming in! The challenge with writing a test is that this particular problem reveals itself only when publishing to a private repository. And, for some reason I can't yet understand, the credentials needed to publish seem to not be available when JavaAppPackaging is enabled together with UniversalDeployPlugin.

Let's have a chat once you are back from holidays! :)

[dotta]

Turns out that if I do a publish before calling universal:publish, then it all works fine. And this is actually a good enough workaround for the moment (at least for my current use case).

[muuki88]

Thanks for sharing :heart:

So you have something like this in your code?

publish in Universal := ((publish in Universal).dependsOn(publish)).value

Maybe we should add this in the the UniversalDeployPlugin as well. Not sure what publish has as a side effect.

[dotta]

I am using sbt-release and we have a +publish before a +universal:publish. But it should be effectively the same as what you wrote.

Not sure what publish has as a side effect.

Same here :)

[asdcdow]

I get the same problem with the rpm plugin. However, running publish beforehand doesn't seem to make it work for me.

[muuki88]

Do you also have a similar setup? A private repo that requires credentials?

[asdcdow]

Yes, I also have a private repo that requires credentials

[asdcdow]

Looking closer at this. I think I'm running in to 2 problems.

• The problem described in the story
• https://github.com/sbt/sbt/issues/2422

I think the second problem is why the workaround is not working for me (I'm trying to upload a >50mb RPM).

I think the problem in this story is cause by the Authenticator not being set for any of the *:publish tasks. The reason this workaround works is because the JVM requires you to set a global Authenticator. So, once it is set by publish then it is set for all of the *:publish tasks as well.

[muuki88]

Thanks a lot for digging deeper into this :heart:

A global Authentiator. This would be a good explanation. I was thinking about the credentials object not being scoped correctly, e.g.

credentials in Rpm := Credentials("..")

So they get picked up during rpm:publish. Is there any way to access this Authenticator instance?

[asdcdow]

I'm not sure how you are supposed to access the authenticator. However, the way I was able to see what was happening was by putting a breakpoint in IvyAuthenticator.install. This gets called before all of the http requests.

[muuki88]

Thanks a lot for sharing. I'm not sure if I have time to debug this. You have already done a lot to shed light into this issue.

Not sure how we handle this from here on. It looks that it's partially a problem in native-packager. If you find a workaround or configuration that fixes this, I'm more than happy to merge it.

[asdcdow]

Due to my other issue, I ended up using sbt-aether-deploy. I was then able to get this working using roughly this stack overflow answer

[dragos]

The workaround using publish before universal:publish doesn't work in Sbt 1.x.

[muuki88]

Thanks for the update 😀

Do you have any time to debug this issues further? I fear I won't even have the time for a minimal setup to do so 😥

[dragos]

I spent already more than a day debugging, and I didn't get anywhere close to a solution. The workaround is to disable Gigahorse (updateOptions in ThisBuild := updateOptions.value.withGigahorse(false)). A few related discussions:

• sbt/sbt#2422
• sonatype support

The usual connection goes like this:

• attempt to connect without credentials
• fail (401 or 403) and retry with credentials

When publishing a SNAPSHOT version this always fails. When publishing a release, it works (NOT with Gigahorse though).

• SNAPSHOTs fail because Sbt (or Ivy, whatever) attempts to do PUT without credentials, and this fails early (broken pipe) because the server, reasonably so, does not wait for the whole upload to finish in order to return a 401. The client has no idea that the reason for the broken pipe is missing credentials, and doesn't try again.
• releases work because Sbt/Ivy first tries to see if the file exists (since you're not allowed to overwrite releases on the server). This first request fails with a well-behaved 401 and Ivy retries with authentication.

Ultimately, I don't think this is a native packager issue, but it arises here more often because uploads are larger (perhaps the server waits and returns 401 for PUTs of certain file types, like pom files, that are small enough).

[muuki88]

Thanks a lot for the detailed explanation and links to related issues :heart: