jvm-repr
jvm-repr copied to clipboard
jupyter
Publish on Maven Central
Right now, jvm-repr is published via jitpack. This requires not only direct users of the library to add jitpack to their repository list, but also transitive users to add it too. (E.g. the scala-interpreter-api of almond now depends on jvm-repr, and its downstream users also have to add jitpack for jvm-repr to be pulled fine.)
jvm-repr could be pushed to Maven Central, like most JVM libraries already do, so that adding the jitpack repo isn't required.
I'd like to take this.
It looks like a TravisCI build based on Gradle is already setup, just need to connect that through to Sonatype.
There's one prior instance of someone publishing from a Travis Jupyter build to Sonatype here, and it's a few years old, so might need to dig into the memory banks of @poplav or @ivanov if there are issues w/ getting those credentials.
@alexarchambault, I tagged 0.4.0 with your latest changes even though this isn't already done. At least you can use them in the mean time.
Created ticket https://issues.sonatype.org/browse/OSSRH-46499 to instantiate a jupyter org, since that's what jvm-repr already uses (as opposed to org.jupyter which already exists, as in declarativewidgets). Fingers crossed! FYI mentioned @rdblue, @rgbkrk, and @alexarchambault in the Sonatype ticket. Hope that's OK.
I believe Maven groupIds are expected to align with Java package names, so org.jupyter is the one we should target. Docs - https://maven.apache.org/guides/mini/guide-naming-conventions.html
There will be no issues claiming org.jupyter given that folks like @rgbkk are contributors to the core project.
The remaining legwork here is setting up a CI build that can publish to Sonatype and securely manage the credentials that allow such publishing. I took a stab at it with Gradle but it was pretty obtuse, requiring several different encrypted key files being checked in to the repo. There's an sbt plugin that seems to more sensibly deal with secret mgmt (don't have the name handy) that I've been meaning to try. But really any solution will do, we don't need a pretty one!
Yes, feel free to claim org.jupyter on Maven for all the jupyter JVM packages.
You've also got support from @mpacer (possibly @parente too 😉 ). Others on the steering council have expressed an interest in us making JVM & Jupyter better, it's just not their domain areas (largely scientific computing in Python, R, and Julia). We are all happy to have you all drive this forward.
I'd like to go ahead with moving jvm-magics into the jupyter org -- have you all checked it out?
There's also some light email about it on the jupyter mailing list -- https://groups.google.com/forum/#!topic/jupyter/t-mXBpKNVJM
That ticket didn't pan out, because I did it incorrectly. But it's a good thing, if org.jupyter is the way to go. Who wants to go about claiming that org? @rgbkrk? @jameskoch? If either of you have a Sonatype account already, that would be the easier way forward – I can make another ticket (properly this time) but it would probably be better coming from someone who's a contributor to the Jupyter project.
I'm in favor of @jameskoch claiming the org, especially since I'll be out on paternity leave for a bit.
What's the status of claiming the org? If the new ticket isn't opened yet, I guess linking to this comment and mentioning that both @rgbkrk and @parente are members of the steering council ought to be fine to claim org.jupyter.
Then about publishing jvm-repr, as discussed with @jameskoch, adding sbt support, then sbt-ci-release, to this repository, would allow to easily publish things from Travis CI. I can handle adding support for both.
Lastly, if we publish from Travis CI, we're going to need:
- a GPG key to sign artifacts,
- sonatype credentials, to push things to Sonatype.
For now, AFAIK, the GPG key isn't checked in any way by Sonatype (they just require the key to be published to a public key server, and artifacts to be signed by such a key). Any throw away key does. A key can be generated, added to the Travis CI settings, then thrown away. And if the settings need to be updated, a new key can be generated. (Yes, it's not too secure overall…)
About the Sonatype credentials, one can get tokens associated to a Sonatype account (as @jameskoch reminded me). A security risk here is that anyone with write rights to this repo can possibly run a job on Travis having access to these credentials, and get them back in one way or another. So it'd be probably better if these came from someone having write access to this repository (@rdblue?), and / or whose credentials don't already have rights to some other sensitive organizations (unlike mine I guess).
The nice thing at least is that with Travis CI we have a very public audit log and can revoke keys afterward.
Then about publishing jvm-repr, as discussed with @jameskoch, adding sbt support, then sbt-ci-release, to this repository, would allow to easily publish things from Travis CI. I can handle adding support for both.
Why does publishing require changing the build system? I would very much prefer not to change the build to SBT. It is not widely used outside of the Scala community and this is not a Scala project.
I was worried this might be contentious... I tried getting Gradle to work here for a day and came away super frustrated. There are few working end-to-end examples of publishing workflows for Travis. There have been recent changes to several of the plugins required, and many examples and docs are out-of-date. On the other hand, in the SBT world, someone has made a plugin that does it all, with minimal hoop-jumping required.
Here's my work-in-progress on Gradle - https://github.com/jameskoch/jvm-repr/commit/dfab9c70105c695673321f2cf641d8ea79c541e1
Big disclaimer, I don't use Gradle regularly, so could be getting in my own way.
Here's what an SBT build would look like - https://github.com/jameskoch/jvm-repr/commit/82aa8e777e419cee0c6f7da57a0b45d88fe60f02
What is the motivation to publish from Travis? Why not register keys for maintainers and publish just releases?
A manual build/publish system would be fine with me. It's a bit less auditable than Travis, and more of a burden on the maintainers, but my primary interest is in Maven Central availability, regardless of how things get there.
Publishing from Travis CI is less of a burden (once it's setup, one just has to tag a release on github, and that's it).
sbt wouldn't necessarily replace gradle, it can be added along it.
Just found a new write-up on Travis+Gradle, updated mid-December - https://github.com/osmlab/atlas/wiki/Gradle,-Travis-CI-and-Maven-Central
It's 21 pages long if printed vs 7 for SBT. Not trying to belabor that point, though, so much as record this as a potential working example that could be achieved w/ Gradle.
I filed https://issues.sonatype.org/browse/OSSRH-46640 to get the ball rolling on giving Alex access to publish to Sonatype.
@rdblue can we get your blessing to merge the SBT file that would let us publish from Travis w/o a ton of work?
Pros:
- Regular users won't need to add custom resolvers
- Lower barrier to entry for enterprises users who might be restricted to common repos
- Public audit log
Cons:
- A second build file that needs to be managed if dependencies are added (though I think we want this project to have basically no dependencies in so far as that's possible).
I don't like the one con, but I went and re-read the Gradle instructions again and just can't imagine it taking less than a half day to grind through the issues.
I don't see value in publishing from a CI tool. I think we should release versions to common repos without complicating this by mixing it with CI. That way we don't have the cons and get all of the pros.
That said, I don't have time to update the build to push releases. If you guys want to maintain the build from here on out, then it is up to you how it gets done.